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Sizing  Distributed  Systems: 
Overview  and  Recommendations 


Sandra  A.  Mamrak 


ABSTRACT 

Computer  system  sizing  is  a  complicated  process  for 
which  a  variety  of  tools  have  been  developed.  The  choice  of 
tools  for  a  particular  sizing  exercise  is  guided  by  many 
considerations  such  as  cost,  available  data  and  the 
expertise  of  the  analyst.  This  report  presents  an  overview 
of  sizing  techniques,  a  brief  discussion  of  the  factors  that 
affect  choosing  one  or  a  combination  of  techniques,  and  a 
set  of  recommendations  for  choosing  tools  for  sizing 
distributed  systems.  The  report  is  aimed  at 
managerial-level  personnel  who  have  developed  technical 
competence  with  regard  to  single-processor  computer  systems 
and  are  faced  with  procurement  decisions  regarding 
distributed  computer  systems  or  services. 


Keywords:    benchmarking;    distributed   systems;    hybrid 
models;   queueing  analysis;   system  sizing. 


1.0   INTRODUCTION 

System  sizing  is  the  process  of  configuring  a  set  of 
computer  hardware  and  software  components  so  that  they  can 
adequately  meet  the  functional  and  capacity  demands  of  a 
given  workload.  It  is  a  complicated  process  because  its 
success  depends  not  only  on  specifying  the  individual 
components  of  both  a  computer  system  and  a  workload,  but 
also  on  capturing  the  myriad  and  as  yet  not  understood 
relationships  among  all  the  components.  Sizing  in 
distributed  computer  systems (1)  is  considerably  more 
complicated  than  sizing  single  computer  systems  because  of 
the  number  and  variety  of  hardware,  software  and  workload 
components  likely  to  be  present. 

Many  techniques  have  been  developed  for  performing 
system  sizing,  ranging  in  sophistication  from  the 
application  of  some  rules-of-thumb  to  the  execution  of 
extensive  benchmark  experiments.   These  techniques  trade  off 


(1)  A  distributed  computer  system  is  defined  to  be  any 
system  in  which  a  set  of  host  computers  or  end  processors, 
terminals,  and  other  peripheral  devices  is  interconnected  by 
way  of  a  communications  subnetwork. 
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expected  accuracy  against  expected  cost,  with  the  most 
accurate  techniques  generally  being  the  most  costly.  Thus, 
a  decision  to  use  one  or  the  other  method  for  system  sizing 
is  largely  a  cost-benefit  analysis,  balancing  the  expected 
accuracy  of  a  given  tool  with  the  budget  available  for  a 
sizing  study. 

Other  factors  besides  cost  and  accuracy  also  affect  the 
choice  of  sizing  tools.  These  factors  include  knowledge  of 
workload,  the  number  of  available  analysts  and  their  level 
of  expertise,  the  scope  of  the  sizing  problem,  the 
availability  of  computer-aided  design  tools,  the 
availability  of  measurement  data,  the  time-frame  for  the 
sizing  study  and  the  credibility  of  the  technique  to  those 
responsible  for  decision-making.  These  factors  often 
dominate  any  "scientific"  considerations  in  choosing  sizing 
tools.  Thus,  cost  and  accuracy  considerations  must  be 
judiciously  balanced  with  a  consideration  of  all  other 
relevant  factors. 

This  report  is  aimed  at  management  personnel  who  are  or 
will  be  faced  with  decisions  about  the  procurement  of 
distribted  computer  systems  or  services.  The  purpose  of  the 
report  is  to  present  a  condensed,  elementary  overview  of 
system  sizing  options,  emphasizing  their  relative  merits 
with  regard  to  sizing  distributed  computer  systems. 

The  next  section  summarizes  system  sizing  techniques  as 
they  are  generally  viewed  by  computer  analysts.  The  summary 
is  presented  primarily  to  establish  a  conceptual  framework 
which  will  facilitate  discussion  in  the  rest  of  the  report. 
Some  techniques  are  discussed  in  more  detail  than  others  to 
set  the  stage  for  recommendations  for  sizing  distributed 
systems.  Section  3.0  presents  a  more  detailed  discussion  of 
factors  other  than  cost  and  accuracy  which  affect  sizing  of 
distributed  systems.  Recommendations  for  sizing  distributed 
systems  are  presented  in  Section  4.0,  based  on  the  issues 
discussed  in  the  previous  two  sections. 


2.0   SYSTEM  SIZING  TECHNIQUES 

In  capacity  planning  and  acquisition  of  computer 
systems,  the  most  fundamental  question  that  must  be  answered 
is  whether  or  not  a  proposed  computer  system  configuration 
will  be  able  to  process  adequately  a  current  or  projected 
workload.  If  a  system  is  under-sized,  the  workload  simply 
will  not  be  able  to  be  processed:  a  disastrous  outcome.  If 
a  system  is  over-sized,  computer  users  are  very  likely  to  be 
paying  for  extra  capacity  which  they  neither  desire  nor  can 
use. 


Adequate  processing  requires  that  a  computer  system  be 
able  to  meet  the  functional  and  capacity  demands  of  a  given 
workload  [GSA79] .  These  demands  may  represent  current  or 
projected  processing  needs.  Functional  demands,  such  as  a 
requirement  to  support  ANS  FORTRAN  or  to  provide  a 
hierarchical  database  system,  can  usually  be  evaluated  in  a 
straight-forward  manner.  Most  often  they  can  be  clearly 
specified  in  a  vendor-independent  language  and  can  be  easily 
assessed  to  everyone's  satisfaction  with  a  yes/no  decision. 
In  contrast,  a  capacity  demand,  or  a  requirement  to  process 
a  workload  in  a  given  period  of  time,  is  much  more  difficult 
to  evaluate. 

The  difficulty  in  evaluating  capacity  demands  stems 
from  two  sources:  1)  the  need  to  represent  complex 
interactions  among  various  computer  hardware  and  software 
components,  and  2)  the  need  to  accrately  represent  a 
projected  workload.  Since  the  performance  of  a  computer 
system  can  only  be  evaluated  with  respect  to  a  given 
workload  [CAL79] ,  an  accurate  model  of  a  projected  workload 
is  essential.  The  ultimate  success  of  system  sizing  depends 
on  how  precisely  the  models  of  both  a  computer  configuration 
and  a  test  workload  represent  their  real  counterparts. 

Table  1  lists  the  range  of  system  sizing  techniques. 
They  are  ranked  in  order  of  increasing  credibility,  accuracy 
and  cost.  Credibility  is  a  subjective  criterion,  varying 
from  analyst  to  analyst.  Accuracy  and  cost  are  difficult  to 
quantify  for  a  given  class  of  techniques.  But  the  ordering 
presented  in  Table  1  reflects  the  relative  ranking  likely  to 
be  assigned  by  practicing  systems  analysts.  A  description 
of  each  sizing  technique  follows. 


Table  1.  System  Sizing  Techniques  in  Order  of  Increasing 
Credibility,  Accuracy  and  Cost 


1.  Subjective  Analysis:  Rules  of  Thumb 

2.  Queueing  Models 

3.  Hybrid  Models:  Queueing,  Simulation 

and  Other  Numerical  Components 

4.  Simulation  Models 

5.  Benchmarking:  Real  System  Running 

Synthetic  Jobs 

6.  Benchmarking:  Real  System  Running 

Real  Jobs 


Subjective  Analysis:   Rules-of -Thumb 

This  technique  involves  the  application  of  reasonable 
" rules-of -thumb"  to  the  system  sizing  problem.  No  formal 
models  are  employed.  Typically,  analysts  will  make 
subjective,  but  informed  judgments  about  what  the  workload 
requirements  are  and  what  hardware/software  configurations 
will  support  them.  These  judgments  are  based  on  personal 
experience  and  on  the  experience  of  other  analysts  who  share 
their  expertise  through  various  publications  or  in  informal 
discussions.  An  example  of  a  rule-of -thumb  that  may  be 
applied  when  sizing  a  database  system  is  the  so-called 
"80-20  rule"  [IDC76].  In  a  decision  about  whether  to 
distribute  or  centralize  a  database,  this  rule  recommends 
centralization  if  80%  or  more  of  database  queries  are  from  a 
local  site  and  20%  or  less  are  from  remote  sites. 

Queue ing  Models 

This  technique  is  characterized  by  the  use  of  formulas 
derived  from  queueing  theory  analyses  of  computer  systems. 
The  systems  are  generally  viewed  as  queueing  network  models 
[HUG73]  with  arrivals  for  service  being  queued  according  to 
various  disciplines  such  as  first-in,  first-out  or  processor 
sharing.  Equations  are  set  up  to  describe  system  behavior. 
Solutions  to  these  system  equations  provide  performance 
quantities  such  as  throughput  rates  and  resource 
utilizations  which  are  useful  for  system  sizing.  Figure  1 
shows  a  typical  queueing  network  model  for  a  batch  load. 

Many  queueing  theory  models  of  computer  systems  exist. 
(A  comprehensive  survey  of  the  analysis  of  queueing  network 
models  of  computing  systems  is  presented  in  the  September 
1978  issue  of  ACM  Computing  Surveys. )  Although  the  modeling 
emphasis  has  been  primarily  on  single  computer  systems, 
computer  communication  networks  have  been  extensively 
studied  [KLE76]  and  some  models  exist  for  distributed 
systems  [BAB77,  LAB77,  MCG78,  WON78] .  A  queueing  model 
developed  for  use  in  sizing  distributed  systems,  which  is 
unique  in  that  it  incorporates  economic  factors,  has  been 
developed  by  Bucci  and  Streeter  [BUC79] . 

Queueing  network  models  that  represent  reality 
faithfully  are  often  not  tractable.  That  is,  if  phenomena 
such  as  multiple  resource  holding,  blocking,  parallel 
processing  and  load  balancing  are  incorporated  in  a  queueing 
model,  then  the  model  cannot  be  analyzed  to  give  exact 
solutions  in  a  reasonably  short  time.  There  has  been 
considerable  study  devoted  to  developing  approximation 
methods  such  as  decomposition  or  diffusion  for  analyzing 
queueing  network  models  of  computing  systems  [CHA78] . 
Although  in  most  cases  it  is  difficult  to  quantify  the  error 
introduced  by  these  approximation  techniques,  they  have 
proved  to  be  very  useful  for  recognizing  and  discarding  poor 
design  choices  (rather   than   obtaining   a   high   degree   of 
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precision  in  predicting  performance  quantities) .  Several 
programming  packages  exist  which  incorporate  queueing 
approximation  techniques  [INF75,  REI78,  SAU77] .  An  example 
which  demonstrates  the  successful  use  of  an  approximation 
technique  is  the  modeling  of  IBM's  Multiple  Virtual  Storage 
operating  system  by  Buzen  [BUZ78] . 

Hybrid  Models 

Hybrid  models  combine  analytic  queueing  components, 
simulation  "  components  and  possible  other  numerical 
techniques.  The  purpose  of  these  hybrid  models  is  to  bridge 
a  rather  wide  gap  between  pure  queueing  models  and  pure 
simulation  models.  Simulation  models  (described  in  more 
detail  below)  have  the  advantage  of  accurately  representing 
complex  interactions,  but  they  can  easily  require  orders  of 
magnitude  more  execution  time  than  a  simpler  analytic 
queueing  model.  Queueing  models  execute  quickly,  but  cannot 
easily  accommodate  complex  interactions.  The  basic  approach 
in  hybrid  modeling  is  to  decompose  a  complex  system  into 
several  subsystems  and  independently  choose  an  appropriate 
modeling  tool  for  each  subsystem  in  an  effort  to  balance 
accuracy  and  speed  tradeoffs. 

The  majority  of  hybrid  models  combine  analytic  and 
queueing  components.  A  noteworthy  exception  to  this  rule  is 
Bard's  hybrid  model  of  IBM  VM/370,  an  interactive, 
multiprogrammed,  virtual  storage  operating  system  [BAR78] . 
In  the  VM/370  Performance  Predictor  standard  methods  of 
queueing  network  analysis  are  supplemented  by  the  use  of  an 
algebraic  transaction  flow  model  and  asymptotic  formaulas 
for  bottleneck  analysis. 

Hybrid  simulation  models  are  useful  when  the  computer 
system  to  be  modeled  can  be  easily  decomposed  into  two 
phases  (see  Figure  2) :  a  phase  of  long-term  resource  usage 
(system  arrival  and  departure  activity)  and  a  phase  of 
short-term  resource  usage  (CPU,  memory  and  I/O  activity) . 

The  first  phase  is  implemented  as  a  simulator,  allowing 
complex  job-arrival  patterns,  arbitrary  scheduling  rules  and 
allocation  policies,  and  multiple  classes  of  jobs.  For 
example,  an  arrival  rate  which  is  dependent  upon  the  time  of 
day  can  be  easily  incorporated  as  a  simulation  feature.  The 
time  units  associated  with  this  phase  are  typically  on  the 
order  of  seconds  or  minutes.  Implementation  of  phase  1  as  a 
simulation  greatly  enhances  the  accuracy  of  a  hybrid  model. 

The  second  phase  is  implemented  as  a  set  of  queueing 
theory  equations.  In  this  phase  time  segments  are  defined 
as  the  time  between  two  successive  job  initiation  or  job 
termination  events  (arrivals  or  departures  to  phase  2) . 
Since  the  composition  of  the  job  mix  in  this  phase  is 
constant  over  a  time  segment,  analytic  prediction  of  time 
segment   performance   is   appropriate.     The    time   units 
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associated  with  this  phase  can  be  as  short  as  microseconds. 
It  is  at  this  level  of  detail  that  simulators  tend  to 
execute  relatively  slowly.  Implementation  of  phase  2  as  an 
analytic  model  significantly  reduces  execution  time  for  the 
hybrid  model. 

Several  languages  exist  for  hybrid  simulations  and 
comparison  investigations  show  them  to  perform  nearly  as 
accurately,  but  in  much  less  time,  than  simulation  models 
alone  [KIM75,  SCH78]  .  An  example  of  the  application  of 
hybrid  simulation  modeling  to  a  complex  computer  system  can 
be  found  in  Browne's  description  of  a  project  to  model  the 
Advanced  Logistics  System  developed  by  the  United  States  Air 
Force  Logistics  Command  [BR075] . 

Simulation  Models 

Discrete-event  simulation  models  represent  computer 
system  activity  as  a  series  of  "events"  and  simulate  running 
of  a  computer  system  by  scheduling,  executing  and  collecting 
data  describing  a  pre-defined  sequence  of  events  over  some 
period  of  time.  The  level  of  abstraction  of  the  events 
depends  on  the  capabilities  of  the  simulation  language  being 
used  and  can  vary  from  "an  arrival  at  a  single  server  queue" 
to  "a  retrieval  request  against  a  hierarchical  database". 
As  mentioned  above,  simulation  models  can  be  used  to 
represent  complex  interactions  at  any  desired  level  of 
detail.  Execution  time  for  a  simulation  is  roughly 
proportional  to  its  level  of  detail. 

Simulations  can  be  written  in  high-level  programming 
languages  or  in  languages  designed  for  general  queueing 
systems  or  even  specific  computer  systems.  A  good  survey  of 
simulation  languages  is  presented  in  SHA75,  Chapter  3. 

Benchmarking:   Real  System  Running  Synthetic  Jobs 

Synthetic  benchmarking  is  a  technique  in  which  the 
system  to  be  sized  is  not  represented  by  a  model,  but  by  an 
actual  hardware/software  configuration.  A  workload  to  drive 
the  system,  however,  is  represented  at  a  fairly  abstract 
level  by  a  set  of  synthetic  tasks  which  are  either  resource 
oriented  [SRE74] ,  or  functionally  oriented  [CON79] . 
Resource  oriented  tasks  are  designed  to  consume  CPU,  memory, 
channel  and  I/O  device  time  rather  than  to  perform 
functionally  (e.g.  do  FORTRAN  compiles  or  text-editing) . 
Functionally  oriented  tasks  are  those  which  perform  some 
pre-defined  automatic  data  processing  function  like  a 
database  query  or  update.  This  sizing  technique  has  an 
advantage  over  previous  methods  in  that  the  difficult  task 
of  modeling  interactions  of  system  components  is  eliminated. 
However,  the  ultimate  success  of  a  sizing  effort  also 
depends  on  the  accuracy  of  the  workload  model  (synthetic 
benchmarks  in  this  case) . 


A  resource-oriented  description  of  a  workload  is  an 
appropriate  one  in  an  environment  where  alternative  systems 
are  essentially  homogeneous  with  respect  to  hardware  and 
software.  It  is  not  appropriate  for  sizing  systems  with 
heterogenous  components,  as  is  very  often  the  case  for 
distributed  computer  systems.  Functionally  oriented 
synthetic  benchmarks  are  valid  for  use  in  heterogeneous 
system  selection.  They  have  been  used  with  apparent  success 
in  some  comuter  procurements  (see  MCN77  for  example) ,  but 
there  is  not  yet  sufficient  data  to  establish  their 
feasibility  in  general.  The  National  Bureau  of  Standards  is 
currently  exploring  the  possibility  of  establishing  a 
central  distribution  facility  for  a  highly  developed  set  of 
synthetic  benchmark  programs  developed  by  the  Department  of 
Agriculture  [CON79] .  If  this  is  done,  use  of  the  benchmark 
materials  could  be  monitored,  thus  providing  a  broader 
database  for  accessing  feasibility. 

Benchmarking:   Real  System  Running  Real  Jobs 

Benchmarking  is  the  most  complex  and  costly  technique 
for  system  sizing,  but  it  is  generally  believed  to  be  the 
most  accurate  method  available  for  sizing  single  computer 
systems.  It  is  the  only  existing  system  sizing  technique 
that,  when  executed  properly,  is  universally  accepted  by 
both  vendors  and  procurement  agencies  as  being  "fair".  In 
this  approach,  as  in  the  case  of  synthetic  benchmarking,  the 
proposed  computer  configuration  is  used  rather  than  a  model 
thereof.  In  addition,  a  complex  model  of  a  test  workload  is 
constructed,  incorporating  functional,  resource-usage  and 
performance  characteristics  of  the  real  workload  [AGR76, 
WRI76] .  Thus,  the  technique  eliminates  as  much  abstract 
modeling  of  the  system  or  workload  as  is  practically 
feasible,  and  incorporates  all  the  complex  interactions 
among  hardware,  software  and  workload  components. 

Benchmarking  is  very  expensive.  Its  cost  can  easily 
reach  millions  of  dollars,  depending  upon  system 
specifications  and  the  number  of  vendors  involved  in  a 
procurement  bid  [PRP78] .  A  large  part  of  the  cost  is  due  to 
the  fact  that  benchmarking  is  labor-intensive  and  can  easily 
occupy  a  highly  skilled  team  of  analysts  for  months.  The 
cost  of  benchmarking  to  size  a  total  distributed  system  is 
expected  to  be  too  high  to  justify  the  benefit. 

There  are  also  technological  problems  that  arise  when 
moving  from  benchmarking  large  single  computer  systems  to 
benchmarking  distributed  systems.  These  stem  from  the 
manner  in  which  benchmark  experiments  are  run.  First,  there 
is  the  need  to  construct  a  workload  which  accurately  models 
a  real  workload.  Although  a  considerable  amount  of  work  has 
been  done  to  guide  test  workload  construction  for  single 
systems  [FER79,  FIP79] ,  no  work  has  even  begun  for 
characterizing  workloads  on  distributed  systems.  Second,  in 
order   to  run  a  benchmark,  the  proposed  hardware/software 


configuration  must  be  assembled  prior  to  actual  purchase. 
Vendors  have  developed  elaborate  benchmarking  centers  which 
allow  assembly  of  components  in  various  combinations  for 
single  system  sizing.  Such  assembly  would  be  extremely 
difficult  for  distributed  systems,  especially  since  most  are 
likely  to  be  composed  of  multi-vendor  components. 


3.0   ADDITIONAL  SIZING  ISSUES 

The  choice  of  techniques  for  system  sizing  is 
influenced  by  a  variety  of  factors  other  than  the  accuracy 
and  cost  of  sizing  tools.  These  other  factors  are  discussed 
in  this  section,  emphasizing  their  potential  impact  on 
decisions  for  sizing  distributed  computer  systems(2). 


3.1   Sizing  Problem:  Scope  And  Frequency  Of  Occurrence 

The  anticipated  complexity  of  a  computing  system  as 
judged  by  1)  the  number  and  kinds  of  possible  alternatives, 
2)  the  expected  frequency  with  which  sizing  decisions  are  to 
be  made  and  3)  the  time  available  in  which  to  do  a  system 
sizing  study  all  impact  a  choice  of  sizing  techniques. 

When  sizing  distributed  systems  the  number  and  kinds  of 
possible  alternatives  will  be  large.  Consideration  of 
hardware  components  alone  presents  choices  among 
telecommunication  carriers,  subnetwork  interface  components, 
host  computers,  user  terminals  and  other  peripheral  devices. 
Various  combinations  of  these  components  provide  possibly 
thousands  of  alternatives  that  have  to  be  compared  for  a 
given  design.  Several  fast,  less  accurate  tools  such  as 
network  queueing  analysis  are  needed  to  eliminate  a  large 
portion  of  unacceptable  alternatives.  After  the  field  is 
narrowed,  more  sophisticated,  but  relatively  slower  tools 
such  as  the  hybrid  modeling  tools  described  above  are 
needed.  Simulation,  and  even  limited  benchmarking  if 
possible,  may  be  employed  for  decision  making  among  a  final 
small  set  of  options. 

The  acquisition  of  a  distributed  computer  system  is 
likely  to  be  an  infrequent  occurrence  for  most 
installations.  Such  major  purchases  are  likely  to  be  made 
once  in  every  five  to  ten  years.  Under  such  circumstances 
it  is  generally  not  cost-effective  for  a  purchasing  agency 
to  spend  large  amounts  of  money  building  up  complex  sizing 
models  and  developing  in-house  expertise  in  the  use  of  the 
models. 


(2)   Chandy   presents   a   similar   discussion   of    factors 
influencing  a  choice  of  analysis  techniques  in  CHA78. 

10 


The  time  allowed  for  any  particular  sizing  study  is 
usually  proportional  to  the  anticipated  size  and  cost  of  the 
proposed  computer  system.  Even  for  a  large  and  complex 
system,  however,  time  for  sizing  studies  is  often  limited  to 
a  few  months  [BR075,  PRP78] .  This  implies  there  is  little 
time  to  build  up  complex  system  models  from  scratch  or  to 
build  up  analyst  expertise  in  using  such  models. 


3.2  Analysts:  Availability,  Expertise  And  Credibility 

The  number  of  analysts  available  for  a  sizing  study  and 
their  level  of  expertise  relative  to  various  sizing  tools 
are  critical  factors  in  a  choice  of  sizing  technqies.  Even 
more  important,  and  perhaps  the  most  critical  factor  of  all, 
is  the  faith  that  management  has  in  the  analysts'  ability  to 
correctly  use  a  set  of  tools  to  solve  its  specific  sizing 
problems. 

The  complexity  of  sizing  distributed  systems  dictates 
that  several  different  kinds  of  tools  be  used.  This  in  turn 
dictates  that  several  analysts  be  available  for  the  sizing 
study.  The  availability  of  a  pool  of  analysts  not  only 
provides  for  diverse  areas  of  expertise,  but  also  allows  for 
partitioning  a  sizing  study  into  components  that  can  be 
studied  in  parallel,  thus  shortening  the  total  time  required 
for  the  study. 

The  attitude  of  management  toward  various  sizing  tools, 
and  the  confidence  that  management  has  in  a  set  of  analysts 
and  their  ability  to  use  those  tools,  strongly  influences  a 
choice  of  sizing  techniques.  Two-way  communication  lines 
must  be  kept  open  between  management  and  an  analysis  team  so 
that  each  understands  the  priorities  and  constraints  under 
which  the  other  is  working.  This  communication  is  an 
absolutely  essential  requirement  if  managers  are  expected  to 
accept  the  recommendations  of  an  analysis  team. 


3.3   Availability  Of  Analysis  Tools 

The  availability  of  computer-aided  tools  is  a  key 
factor  in  the  choice  of  a  sizing  technique.  They  relieve  an 
analyst  of  the  burden  of  developing  such  packages  as  a  part 
of  the  sizing  study  and  they  often  provide  friendly 
interfaces  which  expedite  an  understanding  of  the  underlying 
tools  themselves.  Several  programming  packages  exist  for 
various  sizing  approaches  and  have  been  referenced  in 
Section  2.0. 

A  comprehensive  interactive  program  called  General 
Utility  for  Estimating  System  Size  (GUESS)  has  been 
developed   by   the   Network   Analysis   Corporation   [MCG78] . 
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GUESS  has  been  used  for  determining  the  relative  merits  of 
specific  architectural  alternatives  (i.e.  local  access, 
connection,  switch  and  host  processor  combinations)  given  a 
requirements  specification.  GUESS,  along  with  a  tutorial  on 
its  use,  is  available  to  government  agencies  for  system 
sizing  studies,  but  is  otherwise  proprietary. 


3.4  Availability  Of  Measurement  Data 

The  absence  of  measurement  data  may  preclude  the  use  of 
certain  models  which  require  detailed  descriptions  of  a 
workload  that  can  only  be  obtained  through  measurement.  In 
general,  the  more  sophisticated  the  model,  the  more 
sensitive  it  is  to  the  accuracy  of  input  data.  Thus,  in 
sizing  a  proposed  distribted  system  the  spectrum  of  sizing 
tools  is  likely  to  range  from  simpler  models  which  require 
little  input  information  but  allow  eliminating  bad  choices, 
to  more  sophisticated  models  which  may  even  be  fed  with 
limited  measurement  data  flowing  from  a  prototype  or  early, 
perhaps  reduced,  system  implementation. 


4.0   RECOMMENDATIONS  FOR  SIZING  DISTRIBUTED  SYSTEMS 

The  discussion  of  factors  influencing  sizing  of 
distributed  systems  leads  to  two  fundamental 
recommendations: 

1.  Establish  long-term,  in-house  expertise  in  sizing, 
or  hire  appropriate  outside  experts. 

2.  Develop  a  measurement  center  as  an  integral  part  of 
a  distributed  computing  system. 


4.1   Establishing  In-House  Expertise 

No  "best"  methodology  or  cookbook  approach  is 
appropriate  for  the  general  problem  of  sizing  distributed 
systems.  It  is  a  complex  art,  relying  on  a  set  of 
scientific  tools  that  must  be  used  carefully  and 
intelligently  if  they  are  to  yield  valid  results. 
Experience  in  the  use  of  the  available  tools  is  an  essential 
ingredient  for  sizing  success.  Therefore,  only  a 
knowledgeable,  experienced  analysis  team  will  be  able  to 
properly  size  distributed  systems. 

Large  government  agencies  and  corporations  which  have 
sufficient  resources  available  may  find  it  cost-effective  to 
invest  in  building  up  in-house  analysis  groups   with   skills 
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in  developing  and  using  all  of  the  available  tools  for 
system  sizing.  They  will  all  be  needed  as  the  various 
stages  of  sizing  progress  from  original  "pencil  and  paper" 
analyses  through  full  implementation  and  support  of  an 
operational  system. 

For  those  groups  without  the  resources  to  build  up  and 
maintain  an  in-house  analysis  team,  a  system  sizing  problem 
will  be  best  handled  by  contracting  out  to  an  appropriate 
consulting  firm,  bringing  in  consultants  to  work  with 
on-site  staff,  or  hiring  sizing  experts  for  the  term  of  a 
procurement.  In  the  government,  for  example,  FEDSIM,  the 
Federal  Computer  Performance  Evaluation  and  Simulation 
Center,  could  be  called  upon  to  provide  some  form  of  expert 
assistance,  as  could  other  service  selection  agencies. 
Within  the  normal  time  frame  of  a  typical  sizing  study  it 
will  not  be  possible  to  begin  to  gather  the  required 
personnel  and  tools,  to  build  up  adequate  expertise  and  to 
actually  do  the  sizing.  The  problem  is  simply  too  complex 
and  the  price  of  error  too  high. 


4.2   Developing  A  Measurement  Center 

Distributed  systems,  after  their  initial 
interconnection,  are  likely  to  expand  in  a  modular  fashion, 
on  a  component-by-component  basis.  System  sizing  questions 
will  be  most  often  directed  primarily  at  small  to  medium 
size  host  computers  and  a  variety  of  intelligent  peripheral 
devices.  Sizing  considerations  in  this  environment  can  be 
viewed  as  one  aspect  of  a  comprehensive  "capacity  planning" 
process,  where  capacity  planning  is  defined  as  the 
forecasting  of  future  hardware  and  software  requirements. 

Capacity  planning  is  best  done  by  using  historical 
performance  data  [ART78] .  This  approach  precludes  the  need 
for  using  abstract  models  of  either  a  computer  system  or  a 
workload.  Thus,  performance  evaluation  based  on  historical 
performance  data  has  the  potential  for  very  accurately 
reflecting  the  true  behavior  of  the  system. 

To  be  most  effective,  however,  data  collection  and 
analysis  must  be  carefully  done.  It  is  not  sufficient  to 
amass  a  roomful  of  tapes  which  contain  a  hodgepodge  of 
system  performance  measurements.  A  long  term  commitment  is 
required  to  properly  instrument  distributed  systems,  in 
their  design  phase  if  possible  [NBS78] ,  and  to  establish  and 
maintain  an  extensive  performance  measurement  database 
spanning  several  years  of  measurement.  Questions  of  where, 
when,  what,  why  and  how  to  measure  must  be  studied  in  the 
context  of  how  the  data  can  be  used  to  feed  the  modeling 
[ROS78]  and  benchmarking  [ART78]  activities  that  will  be 
used  to  maintain  and  improve  levels  of  performance. 
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5.0   CONCLUSIONS 

The  choice  of  techniques  used  to  size  distributed 
computer  systems  depends  on  many  factors.  Some  of  these 
factors  introduce  scientific  issues  while  others  are  a 
function  of  the  particular  exigencies  of  a  given  sizing 
problem.  When  all  relevant  factors  are  considered,  it  is 
evident  that  distribted  systems  will  only  be  successfully 
sized  by  an  experienced  analysis  team  that  is  knowledgeable 
in  the  development  and  use  of  a  diverse  set  of  sizing  tools. 
Further,  on-going  sizing  of  distributed  systems  will  be 
greatly  enhanced  by  the  development  and  support  of  a 
measurement  center  as  an  integral  component  of  a  distributed 
computer  system. 
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